The present disclosure relates to processing compute workloads in a distributed computing system, and more specifically to using estimated completion time for a workload to identify one or more computing systems to use for processing a compute workload.
Infrastructure-as-a-service (IAAS) distributed computing systems generally include hardware, software, servers, storage, and other infrastructure components that can be used to process various compute workloads on an on-demand basis. A user generally requests computing resources (e.g., virtual and/or physical machines) from the IAAS system to use for processing a compute workload. An IAAS provider can identify idle resources in the distributed computing system available for use by new compute workloads and assign resources to the compute workload. When the resources assigned to a compute workload complete processing the workload, the resources are generally released and made available for use by new compute workloads.
Individual servers (virtual or physical) within a distributed computing system may have different characteristics (e.g., processing power, memory, power usage, locality to data used in a compute workload, and so on) and/or different compute workloads in a processing queue. Thus, a compute workload may be completed at different times based on the characteristics of the server the IAAS provider assigns for processing the compute workload. For example, a server with a number of large compute workloads in a processing queue may complete a new compute workload at a later time than a server with a small number of compute workloads of the same size (e.g., data processing requirements) or a large number of small compute workloads.